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ABSTRACT.  The  Air  Force  Research  Laboratory,  Air  Vehicles  Directorate ,  Control  Sciences  Division ‘ 

AFRL/V A  C  has  developed  a  process  for  constructing  large-area  3-D  multispectral  terrain  databases  to  support 
Simulation  Based  R&D  (SBR&D)  concept  development  simulation,  research  and  T&E  activities.  The  Multispectral 
database  (MsDB)  development  effort  was  intended  to  investigate  a  database  development  approach  built  from 
multispectral/kyperspectral  imagery  and  elevation  data  that  also  incorporates  material  classification  data  capable 
of  supporting  realistic  out-the-window  visual  and  multispectral  terrain  displays  and  weapons/sensor  models. 
Integrating  the  MsDB  into  these  legacy  models  enabled  us  to  develop  cockpit  representations  that  immerse  the 
warfighter,  scientist,  analyst and  testers  into  a  dynamic  environment  suitable  for  engineering  level  test  and 
evaluation  of  a  multitude  of  systems.  This  paper  covers  integration  of  the  MsDB  with  Paint  the  Night  IR  Scene 
Generation:  integration  with  Mutl\-Mode  Radar  Simulation  (MMRS),  which  provides  Synthetic  Aperture  Radar 
Simulation;  and  integration  with  SubrScene,  which  provides  Qut-The- Window  Scene  generation. 


1.  Technical  Challenges  to  be  Addressed 

In  oidev  to  provide  Air  Force  Research  Laboratory,  Air  Vehicles  Directorate,  Control  Sciences  Division 
(AFRL/V  AC)  a  fully  functional  and  cost-efficient  multispectral  scene  generation  capability,  several  challenges  must 
be  addressed: 

*  AuLliorilalive  representations  of  the  synthetic  natural  environment,  including  geospecific  SEDR1S  transmittals, 
must  be  richly  populated,  sufficiently  documented,  efficiently  organized,  and  possess  sufficient  fidelity  to  meet 
the  requirements  of  scene  generation  systems  which  also  are  li  nked  to  real-time  environmental  databases  and 
HLA-compliant  servers. 

*  State-of  the-art  phenomenological  models  must  be  accessible  to  both  database  generation  processes  and  the  neal- 
time/rmitime  simulation  environment.  This  implies  closer  linkage  between  environmental  data  collection 
processes  and  runtime  environmental  models. 

2,  Team  Experience  and  Capabilities 

Air  Force  Research  Laboratory,  Air  Vehicles  {AFRL/V  A)t  Sensors  (AFRL/SN)  and  Munitions  (AFRL/MN) 
Directorates,  in  conjunction  with  the  U.S.  Army  Redstone  Technical  Test  Center  (RTTC)  and  U.S,  Army 
Communications  and  Electronics  Command  (CECOM)  Night  Vision  and  Electronic  Sensors  Directorate  (NVESD), 
is  currently  developing  a  common  Multispectral  Database  (MsDB)  for  applications  in  AFRL.  The  current  database 
prototype  development  covers  a  2a  x  2°  region,  and  will  be  expanded  incrementally  to  32°.  This  database  will  be 
able  to  support  electro  optical/infrared  (EO/IR),  radar/RF  and  optical  (e.g.  out-the- window)  scenes  at  DTED  Level  2 
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elevation  and  complementary  feature  accuracy  and  resolution;  it  is  also  being  produced  to  support  complementary 
constructive  and  semi-constructive  models  such  as  Joint  Integrated  Mission  Model  (JTMM).  JSAF  and  OneSAF 
Testbed.  AFRL/V  A,  RTTC,  and  NVESD  have  signed  a  Memorandum  of  Agreement  (Mo  A)  that  go  vems 
collaboration  on  this  and  related  projects, 

3,  Advanced  Concepts 

The  AFRL  team  is  also  conducting  collaborative  research  into  high-resolution  real-time  graphics  solutions  on  low- 
cost  PC  platforms,  as  well  as  mid-to-high  cost  massively  parallel  approaches  to  real  time  multispectral  sensor 
simulation.  In  addition,  research  continues  on  development,  integration  and  evolution  of  real-time  simulation 
technologies  such  as  model-to-hardware  interfaces,  closed-loop  simulation  technologies,  and  improved  Radio 
Frequency  simulation  fidelity.  Ail  three  partners  have  operational  real-time  sensor  scene  generation  systems  based 
on  Silicon  Graphics™  Onyx  Reality  Engine  platforms,  which  support  12  to  48  bit  texture  rendering  and  a  system 
bandwidth  ranging  from  11  716  gigabytes  per  second.  The  NVESD  Paint  the  Night  EQ/IR  scene  generation  package 
is  one  of  die  AFRL  team  systems  that  make  full  use  of  these  high-performance  computational  assets.  Due  to  recent 
technological  advances  in  low-cost  (hundreds  to  thousands  of  dollars)  PC  graphics  acceleration  hardware,  high- 
performance  computing  systems  based  on  Linux  PC  (e.g.  “Beowulf1 F)  clusters*  and  network  bandwidth,  the  AFRL 
team  is  exploring  the  scene  generation  trade  space  to  determine  which  hardware  and  software  solutions  offer  the  best 
value  in  terms  of  scene  fidelity,  real-time  performance,  adaptability  and  ease  of  development.  Innovati  ve  rendering 
technologies,  including  ray  tracing  methods,  are  being  explored  together  with  traditional  textured  polygon  rendering 
methods,  in  order  to  enhance  the  ability  of  graphics  engines  improve  the  robust  fidelity  of  specific  environmental 
phenomena.  These  include,  but  are  not  limited  to  such  optical  effects  as  light  scattering,  shadows,  reflection, 
turbulence  and  an  order  of  magnitude  increase  in  emissive  energy  sources  available  to  the  scene  generation  system. 
The  AFRL  team  has  a  strong  commitment  to  make  best  use  of  DM  SO  products  and  standards,  including  HLA  and 
SEDRIS,  in  conjunction  with  these  initiatives. 

4.  Multispectra]  Database  (MsDB)  Development  Process 

4.1  Overview 

Our  large  area  3D  muUbspeetral  terrain  database  is  fundamentally  composed  of  remotely  sensed  multi-spectral 
imagery,  ground  elevation,  and  material  classification  feature  data.  The  U.S,  Army  Redstone  Technical  Test  Center 
(RTTC)  is  responsible  for  defining  requirements  for  and  initially  processing  this  data  for  further  integration  into  a 
run-time  digital  terrain  database.  In  the  following  sections,  we  discuss  the  requirements  definition^  image 
processing  workflowT  and  the  imagery  derived  products  produced  by  RTTC. 

4.2  Requirements  Definition 

We  begin  the  MsDB  development  process  by  defining  the  imagery,  elevation,  and  material  classification  feature 
requirements.  These  requirements  will  determine  the  input  data  we  acquire  and  the  processes  we  implement  to  meet 
the  goals  of  this  particular  effort.  We  first  determine  source  imagery  from  which  to  develop  our  database.  Wc 
consider  a  number  of  factors  as  we  determine  the  imagery  source.  These  factors,  among  others,  include: 

*  Security  classification  of  database 

*  Necessary  imagery  spatial  and  spectral  resolution 

*  Region  of  interest  land  area  size 

*  Hard  disk  space  necessary  for  imagery 

*  Data  availability  and  acquisition  costs  for  imagery 

*  Processing  costs  for  imagery 

*  Target  usage  of  the  terrain  database 

In  the  following  section  we  provide  a  brief  overview  of  these  factors  that  help  us  determine  our  final  processing 
workflow.  The  desired  security  classification  for  this  project  is  UNCLASSIFIED,  We  can  process  and  extract 
information  from  CLASSIFIED  imagery  sonrees.  However,  we  consider  a  core  set  of  UNCLASSIFIED  remote 
sensing  data  from  which  to  choose  for  this  effort.  This  set  includes  Ikonos,  Quickbird,  SFGT5,  and  La  rid  sat  7 
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ETM+,  This  is  not  an  exhaustive  list  of  sources.  However,  based  upon  previous  research  and  production  experience 
we  believe  these  are  the  most  appropriate  options  for  this  application.  It  should  be  stated  that  we  have  examined  the 
efficacy  of  integrating  Controlled  Image  Base  (CIB)  data  from  NIMA.  Though  we  can  readily  ingest  and  utilize 
C1B  data,  the  image  tonal  quality  is  not  suited  for  our  final  product.  Therefore  we  have  not  included  this  as  a 
possibie  data  source  for  the  purposes  of  this  paper.  In  the  following  section  we  provide  a  brief  description  of  the 
sensors  we  have  chosen  to  consider. 


Ikonos  is  a  commercial  satellite  launched  in  September  1 999  by  Space  Imaging,  Inc.  Ikonos  imagery  is  available  in 
various  formats  and  processing  levels.  The  basic  data  specifications  for  Ikonos  imagery  is: 
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Quickbird  is  a  commercial  satellite  bunched  in  October  2001  by  Digital  Globe,  Inc.  Quickbird  imagery  is  available 
in  various  formats  and  processing  levels.  The  basic  data  specifications  for  Quickbird  imagery  are: 
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SPOTS  is  a  commercial  satellite  launched  in  May  2002  by  SPOT  linage,  SPOT5  imagery  is  available  in  DIMAP 
format  and  various  processing  levels.  The  basic  data  specifications  for  SPGT5  imagery  are: 
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The  Landsat  program  is  a  joint  initiative  between  the  United  States  Geological  Survey  (USGS)  and  the  National 
Aeronautics  and  Space  Administration  (NASA)  which  began  in  the  early  19705s.  The  Landsat  7  satellite  was 
launched  in  April  1990.  Landsat  7  ETM+  is  available  in  various  formats  and  processing  levels.  The  basic  data 
specifications  for  Landsat  7  ETM+  imagery  are: 
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From  these  sources  we  may  choose  from  spatial  resolutions  varying  from  0.61  m  to  60m,  Each  sensor  offers  a 
panchromatic  band  in  addition  to  at  4  multi-spectral  bands.  These  multi-spectral  bands  will  provide  information  for 
extracting  vegetation  and  other  material  signatures. 

Our  overall  goal  is  to  produce  a  large  area  terrain  database  to  include  a  land  area  of  32*.  This  area  encompasses 
approximately  7&D,GG0km2.  The  bounding  box  for  this  area  is  shown  in  figure  1. 
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Figure  1 :  Map  yf  entire  32°  3D  terrain  database  region. 

Wc  will  begin  by  developing  a  2D  k  2a  prototype  region,  and  then  incrementally  build!  outward  Lo  until  the  entire  32& 
is  complete. 

Disk  space  is  not  a  trivial  issue  when  dealing  with  imagery.  We  must  consider  tins  to  ensure  we  establish  the  proper 
processing  infrastructure  to  efficiently  work  with  large  imagery  sets.  As  stated  previously,  we  have  defined  an 
overall  database  are  of -7  SO, 000km2.  The  following  table  states  the  approximate  disk  space  needed  to  cover  this 
entire  area  using  the  source  imagery  we  are  considering. 


SeiEOf  (RcsoIiUkml  #  BhtkIs} 

Quickbirci  <0.61  n\  4) 

fksmos  (1m,  4)* 
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Disk  Space  (G  B) 
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*  1 1  bh  data  All  otlier  dal a  is  8  bit  data 


Acquisition  cost  for  imagery  is  another  factor  we  must  consider.  These  remote  sensing  platforms  provide  valuable 
information  which  is  reflected  in  the  purchase  price.  The  following  table  states  the  imagery  acquisition  costs  to 
cover  the  entire  32a  area. 
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5  J  05. 

The  acquisition  costs  for  this  imagery  gives  rise  to  the  availability  of  this  data.  Although  it  is  possible  to  purchase 
this  data  directly  from  the  various  providers*  it  is  desirable  to  reuse  valid  imagery  sets  already  available  to  DoD 
projects.  By  working  wc  the  National  Imagery  and  Mapping  Agency  (N1MA)  we  are  able  to  search  the  Commercial 
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Satellite  Imagery  Library  (CSIL)  archive  and  request  available  data  at  no  cost  and  utilize  such  data  for  the 
development  of  our  3D  multi -spectral  terrain  database.  This  allows  ns  to  attenuate  the  acquisition  costs  to  a  great 
extent. 

The  target  usage  of  the  terrain  database  is  for  simulation,  training,  and  experimentation.  We  arc  not  developing  a 
database  intended  for  targeting  purposes.  Though  we  process  The  input  data  to  achieve  accurate  registration,  we  are 
not  constrained  by  rigid  accuracy  requirements  such  as  are  needed  for  targeting  purposes. 

Now  that  we  have  briefly  described  the  factors  For  determining  imagery  sources,  we  move  to  a  discussion  of  the 
elevation  and  feature  data.  The  requirement  lias  been  established  that  we  use  Digital  Terrain  Elevation  Data 
(DTED)  level  2  elevation  data.  DTED  2  data  is  ground  earth  elevation  data  with  a  post  spacing  of -30m,  DTED  data 
is  available  from  MIMA  over  a  large  portion  of  the  globe.  However,  DTED  level  2  complete  global  coverage  is  not 
fully  a  vai  lable  yet.  Therefore  we  consider  alternative  processing  methods  of  DTED  level  1  data  to  reach  the  desired 
post  spacing  of  30m.  There  arc  other  options  for  elevation  data,  which  we  will  discuss  later. 

Finally,  we  consider  die  feature  data  that  we  must  extract.  The  requirements  set  forth  for  this  effort  are  to  extract: 

*  Roads 

*  Rivers 

*  Area  bodies  of  water 

*  Building  footprints 

*  Railroads 

*  Vegetation 

There  are  a  number  of  feature  sets  already  available  over  areas  of  the  globe  that  include  these  features  with  varying 
levels  of  material  attributes,  NIMA  provides  VMAP  data  over  areas  of  the  globe  at  varying  resolutions  as  one 
example.  However,  these  data  sets  are  not  fully  available  with  global  coverage,  and  these  features  may  not  be 
precisely  correlated  to  rectified  high-resolution  imagery.  Therefore,  we  look  to  extracting  new  feature  sets  based 
upon  medium  to  high-resol  udon  remote  sensing  data.  We  can  spatially  extract  the  aforementioned  features  from 
panchromatic  or  visible  color  imagery  using  manua]  and  semi-autonomous  extraction  methods.  However,  we  must 
utilize  multi-spectral  data  to  extract  material  classification  information  for  these  features.  Therefore,  we  must 
identify  an  appropriate  imagery  source  containing  multi -spectra]  information,  and  not  simply  panchromatic  or 
visible  color. 

Based  upon  these  factors  we  have  established  a  set  of  requirements  from  which  be  begin  our  development.  We 
begin  by  establishing  a  foundation  imagery  base  using  Landsat  7  ETM+  imagery,  Using  this  imagery  we  create  a 
15m,  pan-sharpened,  multi-spectral  imagery  mosaic  over  the  2°  x  2D  prototype  region  of  interest.  From  this  data  we 
extract  features  with  their  material  properties  using  this  imagery.  W  e  have  examined  existing  feature  data  available 
over  this  region  and  have  determined  to  incorporate  VMAP  Level  1  data  only  over  areas  over  which  we  do  not  have 
sufficient  multi-spectral  imagery  coverage.  Once  we  complete  die  entire  Landsat  7  ETM+  processing  stage,  we 
identify  high  priority  areas  within  this  region  and  incorporate  2.5m  SPOT  imagery.  Using  this  imagery  we  extract 
higher  spatial  resolution  features  that  are  correlated  with  the  features  previously  extracted  from  the  Landsat  data. 

For  the  elevation  data  we  utilize  existing  DTED  level  2  data,  where  available.  Where  DTED  level  2  data  is  not 
available,  we  resample  DTED  level  1  data  to  reach  the  desired  post  spacing  of -30m 

We  have  defined  our  initial  requirements  list  and  are  ready  to  begin  processing  the  data.  This  requirements  list  may 
change  throughout  the  process  as  newr  data  sources  are  made  available.  However,  the  same  basic  workflow  will  still 
apply. 

4.3  linage  Processing  Workflow 

We  begin  the  image  processing  workflow  by  acquiring  the  necessary  imagery,  elevation,  and  existing  feature  data. 
We  request  DTED  and  VMAP  feature  data  directly  from  NIMA.  We  search  the  CSIL  archives  for  existing  imagery 
over  the  entire  32a  region.  Though  we  are  initially  focusing  on  the  development  of  a  2&x2°  area,  it  is  most  efficient 
to  search  the  entire  area  at  one  time  in  order  to  generate  a  data  coverage  map  for  future  reference.  From  our  initial 
query  and  request,  we  acquire  sufficient  Landsat  7  coverage  from  the  CSIL  over  AQ[  #1 .  We  begin  by  examining 
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the  imagery  and  performing  rectification  when  needed.  Once  the  images  are  properly  rectified,  we  mosaic  and 
tonally  balance  the  imagery.  We  first  mosaic  the  panchromatic  bands,  then  the  30m  multi -spec Lral  bands  separately, 
Figure  2  shows  the  tonally  unbalanced  input  imagery  we  have  over  this  particular  region. 


Figure  2:  Tonally  unbalanced  image  mosaic  displaying  image  quality  prior  to  histogram  processing  and 

feathering. 


This  sample  mosaic  displays  the  clouds,  .seasonable,  and  other  atmospheric  differences  that  must  be  attenuated  or 
eliminated  to  produce  a  suitable  seamless  image  mosaic.  It  should  be  stated  that  the  ideal  solution  to  these 
obscurations  is  to  acquire  a  series  of  images  collected  at  or  near  the  same  time  under  cloud-free  conditions.  We  take 
this  approach  later  as  we  integrate  higher  resolution  SPOT5  imagery.  However,  at  this  point  wre  are  simply  laying 
down  a  low-resolution  foundation,  over  which  we  will  mosaic  higher  resolution  imagery  incrementally  as  the  project 
continues.  Therefore,  we  simply  attenuate  these  obscurants  at  this  point  and  move  on  to  mosaicking  the  imagery. 

We  First  attenuate  the  color  differences  by  altering  the  histograms  of  each  color  channel  to  match  the  adjacent  image. 
We  implement  a  form  of  histogram  specification  to  achieve  these  desired  results.  We  then  utilize  cutline  feathering 
to  eliminate  the  presence  of  clouds  as  much  as  possible.  The  final  result  is  a  virtually  seamless  image  mosaic  shown 
in  Figure  3, 
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Figure  3:  Tonally  balanced  image  mosaic  alter  histogram  specification  and  cut  line  feathering. 

This  color  mosaic  is  composed  of  three  separate  Landsat  7  scenes  collected  in  May  and  October  of  the  same  year,. 
We  mosaic  and  tonally  balance  the  15m  panchromatic  and  30m  multi-spectral  bands  separately.  Finally  we 
implement  a  pan-sharpening  algorithm  to  generate  a  louvre  solution  natural  color  image.  The  final  products  of  this 
stage  are  a  tonally  balanced  6  band,  30m  resolution  multi-spectral  image  to  be  used  for  feature  extraction  and  a 
tonally  balanced  15m  natural  color  image  used  to  directly  feed  the  QTW  visualization  and  to  spatially  enhance  the 
feature  extraction  from  the  30m  resolution  data. 

Our  feature  extraction  goals  for  the  2ax2*  prototype  region  include  some  basic  features  listed  previously.  We  begin 
by  first  identifying  vegetation.  Using  die  6  band  multi-spectral  mosaic,  we  implement  a  series  of  supervised  pattern 
classification  algorithms  lo  identify  vegetation  present  in  the  scene.  Then  we  identify  the  vegetation  type  by 
matching  the  spectral  signatures  of  the  extracted  classes  with  existing  spectra  l  libraries  we  have  resampled  to 
correlate  with  the  wavelengths  consistent  with  Landsat  7  ETM+  data.  Next  we  identify  water  features  such  as 
rivers,  lakes,  and  ocean  boundaries.  We  initially  extract  these  features  using  supervised  classification  algorithms, 
and  then  refine  the  extracted  boundaries  manually.  We  then  identify  road  and  railroad  linear  features  in  the  scene. 
Although  some  supervised  classification  algorithms  may  be  used  for  extracting  these  features,  it  is  more  efficient  to 
manually  extract  these  features  then  perform  spectral  matching  lo  identify  the  material  type  for  the  roads.  Finally 
we  manually  identify  building  footprints  in  the  imagery.  Due  to  the  relatively  low  spatial  resolution  of  Landsat 
imagery,  15m,  we  are  unable  to  identify  exact  building  footprints  for  an  average  sized  building.  Therefore  we 
identify  the  bounding  boxes  for  built-up  areas  containing  multiple  buildings.  When  possible,  we  extract  building 
footprints  for  extremely  large  buildings  as  well.  Using  higher  resolution  imagery,  we  may  extract  building 
footprints,  and  we  discuss  this  later  in  the  paper.  An  example  of  our  feature  extraction  is  shown  in  Figure  4. 
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Figure  4;  Overview  ot  g^oref^renced  extracted  features. 


Once  we  have  extracted  these  features,  we  assign  attributes  to  them.  For  tlie  respective  features  we  provide 
vegetation  typen  soil  type,  road  surface  type,,  road  width,  and  river  width.  We  then  export  these  feature  vectors  to 
attributes  2D  Environmental  Systems  Research  Institute  (ESRI)  MhapefIlesP  Tiie  features  arc  then  used  further  in  the 
synthetic  scene  generation  process. 

Finally,  we  resample  the  available  elevation  data  to  30m  post  spacing.  After  inspecting  the  final  elevation  data,  we 
export  it  to  usable  format  for  furLhcr  integration.  At  this  time  we  use  a  GeoTiff  format  for  the  elevation  data.  We 
may  also  export  the  data  into  a  variety  of  standard  formats  such  as  DIED,  USGS  Digital  Elevation  Model  (DEM), 
or  other  suitable  georeferenced  binary  format.  These  formats  depend  upon  tlie  needs  of  the  end  user. 

In  summary  tlie  final  output  products  for  our  initial  work  using  Landsat  data  arc: 

*  1 5m  pan-sharpened  natural  color  image  in  GeoTiff  or  NITF  format 

*  30m  post  spacing  elevation  data  set  in  GeoTiff,  DTED,  DEM,  or  NITF  format 

*  Attributes  feature  set  in  Shape  file  formal  including 

*  Built-up  areas 

*  Roads  S:.  railroads 

*  Rivers 

*  Lakes 

*  Ocean 

*  Vegetation 

The  products  may  be  reused  for  integration  with  other  terrain  database  efforts.  We  also  maintain  foe  original 
imagery  and  elevation  data  for  further  updating  and  rectification  as  needed.  Therefore  when  new  imagery  is 
collected  over  these  areas,  wre  can  rapidly  integrate  this  data  into  the  existing  mosaic  and  generate  a  new  tonally 
balanced  mosaic  tor  further  reuse.  One  underlying  goal  in  this  effort  is  to  produce  imagery  derived  products  that 
maybe  widely  reused.  In  order  to  achieve  this,  we  must  provide  the  data  is  a  wide  array  of  formats.  We  have 
previously  presented  specific  formats  in  which  we  deliver  the  data  to  NVL  and  AFRL.  However,  we  have  die 
ability  to  provide  this  data  in  a  number  of  other  formats  to  meet  customer  requirements.  As  we  proceed  in  this 
effort,  we  will  define  the  most  needed  formats  and  ensure  our  ability  to  export  efficiently  to  those  formats. 
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To  this  point  wo  have  provided  a  bask  overview  of  the  image  processing  workflow  using  Landsat  7  ETM+  imagery, 
Using  this  data  we  have  generated  an  image  foundation  with  a  spatial  resolution  of  15m,  We  now  proceed  to 
integrating  newly  acquired  SPOT5  imagery  into  the  scene  to  reach  a  higher  spatial  resolution, 

4.4  Higher  resolution  imagery  integration 

At  this  time  we  have  chosen  to  acquire  SPOTS  data  instead  of  Quickbird  and  Ikonos  for  higher  resolution  imagery 
work  because  it  provides  the  best  tradeoff  between  resolution,  spatial  footprint,  and  price.  We  are  able  to  acquire  a 
60km  x  60km  SPOT5  bundle  for  about  one  tenth  of  the  cost  of  Quickbird  or  Ikonos  imagery.  This  bundle  contains  a 
2.5m  resolution  panchromatic  band,  three  10m  resolution  multi-spectra)  bands,  and  a  20m-resolutiun  short  wave  1R 
band.  We  request  SPOT  5  Level  1A  processed  imagery.  Level  1A  imagery  has  been  previously  corrected  by 
normalizing  CCD  response  to  compensate  for  radiometric  variations  due  to  detector  sensitivity. 

Once  wc  have  acquired  a  SPOT  5  scene,  we  begin  by  geometrically  rectifying  the  panchromatic  and  multi-spectral 
bands.  We  then  generate  a  natural  color  image  using  the  red,  green,  and  near  IR  bands  in  the  image,  SPOT  5  does 
not  produce  a  blue  band  as  other  sensors  do.  Using  the  bands  supplied  the  image  appears  as  a  false  color  image,  as 
shown  in  Figure  5. 


Figure  5:  False  color  view  of  input  SPOT  5  imagery  without  blue  spectral  band. 


In  order  to  generate  a  natural  color  image  using  this  data  we  must  implement  a  spectral  band  transformation.  This 
transformation  is  given  below 


Rn  =  0.9(RJ}  +  0.  l(NIRr) 
Gn  =  0.7(G')  +  0.3(N1R’) 
Bn  -G' 


s 


Where:  Rn  =  Natural  Color  Red  Band 

Gn  =  Natural  Color  Green  Band 
Bn  =  Natural  Color  Blue  Band 

R’  =  Input  SPOT  5  Red  Band 
G*=  Input  SPOT  5  Green  Band 
B*  =  Input  SPOT  5  Blue  Band 

Implementing  this  transform  produces  Lhe  natural  color  image  shown  in  the  figure  below. 


Figure  6:  Natural  color  view  of  SPOT  5  imagery  after  transformation. 

Once  we  have  rectified  the  SPOT  5  imagery  and  have  created  a  natural  color  image,  we  are  ready  to  refine  the 
features  in  the  scene,  In  this  case  we  have  already  extracted  features  over  this  scene  using  the  Landsat  7  ETM+ 
imagery.  These  features  may  now  be  overlaid  onto  the  rectified  SPOT  5  scene.  We  then  examine:  the  features  to 
determine  whether  they  need,  to  be  spatially  refinedr  If  so,  we  update  the  features  and  save  these  refined  features. 
Because  we  ean  identify  smaller  features  using  the  higher  resolution  SPOT  5  imagery,  we  then  augment  the  feature 
set  based  upon  this  information.  As  we  receive  more  SPOT  5  imagery,  we  repeat  the  process  and  ensure  feature 
alignment  across  the  image  scams.  We  also  ensure  tonal  balancing  between  the  low  resolution  Lands  at  scenes  and 
the  high  resolution  SPOT  scenes.  Finally,  wc  export  the  Landsat  and  SPOT  5  scenes  separately*  the  attributed 
feature  data  and  the  elevation  data  for  further  integration  into  the  scene  generation  process. 

5,  Integration  with  Faint  the  Night  (FTN)  IK  Scene  Generation 

Paint  the  Night  (FTN)  is  a  real-time  infrared  imaging  sensor  simulation  allowing  the  user  to  view  scenes  in  a  3-D 
virtual  world  as  if  through  a  real  or  notional  imaging  sensor,  PIN  was  developed  initially  to  aid  the  Army  in:  sensor 
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design,  prototyping,  and  analysis;  search  and  target  acquisition  mode]  development;  evaluation  o f  tactics*  techniques 
and  procedures;  and  automatic  target  recognition  training  and  analysis.  Because  FTN  was  designed  primarily  for  the 
ground  or  low  altitude  surveillance  tasks  critical  to  the  Army  mission;  PTN  represents  all  terrain  objects  as  discrete 
geometry  and  does  not  utilize  overhead  imagery  directly  as  textures.  Terrain  objects  include  roads*  rivers,  trees*  and 
buildings.  Ail  are  represented  as  3-D  geometry  and  attributed  with  material  classifications  and  geo-typical  textures. 

ln  order  to  generate  the  virtual  representation  of  a  geo-spec  itic  area,  PTN  must  be  provided  geographic  information 
about  Lbat  area  in  the  form  of  3-D  geometry  along  with  rendering  information  for  that  geometry.  The  format  that 
information  is  stored  in  at  runtime  effects  the  performance  of  the  simulation.  It  is  therefore  necessary  to  compile 
and  pre-process  that  data  into  an  efficient  runtime  database  prior  to  execution.  Our  terrain  build  process  takes  GlS 
information  about  a  geo-specific  area  and  convert  it  into  the  required  3-D  formats  for  Paint  the  Night,  called  a  P  IN 
terrain  database. 

Because  PTN  functions  in  concert  with  a  SAF  (Semi-automated  Force),  used  to  coordinate  vehicle  movements,  the 
terra  in  representations  used  by  each  must  be  correlated,  file  SAP  requires  s  iniilar  geo-spec  ific  information*  which  it 
uses  to  determine  movement  and  inter  visibility  of  the  military  units  involved  in  its  simulated  war- fighting  exercise. 
To  ensure  the  necessary  correlation  between  SAP  terrain  and  PTN  terrain  both  representation  should  be  created  from 
the  same  source  data  and  using  the  same  approximations.  Our  build  process  for  the  terrain  takes  the  same  GIS 
information  and  creates  a  SAF  terrain  database. 


Figure  7:  PIN  Terrain  Build  Process, 

In  this  effort,  we  are  taking  the  MsDB  data  and  converting  it  to  PTN  and  SAF  runtime  database.  Feature 
information  in  the  MsDB  will  be  in  E5RI  Shape  format  and  the  elevation  information  in  DEM  format.  The  imagery 
in  the  MsDB  will  only  be  used  as  a  reference.  Our  existing  tools  arc  functional  but  are  limited  in  their  ease  of  use 
and  inefficient  for  manipulating  large-scale  databases  such  as  the  one  required  for  this  project.  In  preparation  lor 
the  database  build  Night  Vision  is  upgrading  these  tools.  The  dataflow  is  shown  in  Figure  7  followed  by  an 
explanation  of  these  upgrades. 

This  process  is  based  on  utilizing  commercial  tools  wherever  available  and  a  redesigned  terrainGcn  (P  I  N  terrain 
generation  code).  A  conceptual  drawing  of  the  new  terrainGen  code  is  depicted  in  figure  8.  This  new  design 
organizes  the  terrain  generation  functions  into  a  core  process  with  plugins  for  input  and  output.  All  the  necessary 
geometric  manipulations  occur  in  the  core  process  to  ensure  that  the  terrain  formats  remain  correlated.  We  are  only 
planning  to  support  a  single  input  format  for  elevation  and  feature  data.  The  feature  input  format  will  be  Shape  to 


Correspond  with  the  MsDB.  Our  input  elevation  data  will  be  in  a  ra  w  binary  format  with  file  format  and  geographic 
information  stored  in  a  separate  XML  flic  in  order  to  achieve  maximum  flexibility.  The  terrain  builder  is  required  to 
convert  other  input  data  forma  ts  to  our  standards  and  do  additional  manipulations  or  projections  using  Arc  View 
and/or  Imagine.  We  plan  to  output  only  our  virtual  (pfb)  and  the  one  constructive  format  (ctdb)  initially,  but  if  a 
format  changes  or  another  one  is  added  it  will  be  much  easier  to  support  it  using  this  architecture.  rIhe  other  major 
ehangc  is  the  incorporation  of  a  GUI,  this  will  not  be  simply  a  cosmetic  change,  by  making  the  process  more 
intuitive  we  should  be  able  to  reduce  the  errors  in  the  build  process  and  cut  our  rebuild  time. 
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Figure  8;  Conceptual  Design  of  the  New  terrainGen  Code. 

In  addition  to  generating  our  FI  N  databases  and  SAF  database,  we  are  also  providing  3-D  geometry  to  Boeing  in 
OpcnFlight  format  lor  conversion  and  use  in  MMRS.  The  OpenFlight  files  arc  created  by  importing  our  PTN  pfb 
files  into  Alias|  Wave  front’s  Maya  using  an  in-house  pfb  reader  plugin  and  outputting  OpenFlight  files. 


6.  Integration  with  Multi-Mode  Radar  Simulation  (MMRS)  Synthetic  Aperture  Radar 
(SAR)  Generation 

The  AFRL  team  made  considerable  progress  integrating  the  simulators  source  database  (SDB)  into  the  MMRS.  The 
SDB  here  is  defined  as  the  database  derived  from  raw  data  sources  such  as  high  altitude  multi-spectral  imagery, 
NIMA  data,  etc.  One  of  the  end  objectives  of  this  effort  is  to  define  a  complementary  MMRS  /  SDB  architecture 
such  that  the  SDB  will  automatically  format,  using  the  MMRS  Formatter  software,  into  a  MMRS  runtime  database 
without  any  human  intervention.  This  complementary  architecture  would  maximize  the  fidelity  and  data  content  of 
the  simulated  radar  imagery.  The  team  is  mutually  working  together  and  making  changes  in  cither  die  SDB  or  the 
MMRS  /  MMRS  formatter  that  makes  the  most  sense  that  best  achieves  these  goals. 


For  example,  one  of  the  architectural  issues  addressed  was  the  definition  of  trees  in  the  SDB.  The  SDR  contained 
triangles  textured  with  images  of  trees  without  additional  information.  Under  these  circumstances  the  MMRS  would 
process  the  geometry  of  the  vertical  plane  surfaces,  or  none  if  viewed  edge  on,  into  an  unrealistic  radar  return.  Once 
this  anomaly  was  identified  it  became  evident  that  by  adding  additional  information  to  the  tree  references  such  as  the 
type  of  tree,  size,  etc.  that  the  MMRS  formatter  software  would  automatically  replace  the  polygon  with  high  fidelity 
radar  tree  models.  The  MMRS  would  then  automatically  process  these  models  to  such  effects  as  seasonal  changes; 
translucency  effects  resulting  from  different  transmitted  frequencies,  etc. 

Progress  was  also  made  by  expanding  the  MMRS  formatter  to  accommodate  different,  but  correct,  OpenFlight 
building  model  conventions  used  in  the  SDB  than  traditionally  formatted  by  the  MMRS. 

Other  issue  currently  in  work  is  how  best  to  refine  elevation  anomalies  between  ocean  and  terrain  boundaries.  Most, 
if  not  alf  commercial  terrain  polygon  rendering  tools  result  in  shorelines  with  water  having  terrain  elevation 
polygons.  Depending  on  radar  mode,  aircraft  elevation,  look  angle,  these  differences  can  result  in  incorrect  and 
misleading  radar  imagery, 

The  MMRS  was  also  modified  to  support  new  versions  of  Linux.  Unfortunately  MMRS  runtime  libraries  compiled 
under  one  version  of  Linux  needed  to  have  its  Make  Files  reworked  in  order  to  compile  under  a  newer  Linux 
version, 

7.  Integration  with  SubrScene  Out-The- Window  Generation 

SubrScene  provides  a  visual  interface  or  scene  generator  for  the  QTW  database.  As  the  MsDB  is  designed  to  fit 
several  needs,  modifications  to  the  data  received  from  RTTC  and  NYL  are  needed.  The  visual  portion  of  the 
database  is  focused  on  fast  moving  aircraft  above  10,000  ft,  while  databases  for  PTN  and  MMRS  are  being 
optimized  for  IR  and  radar.  Thus,  the  database  for  OTW  is  being  optimized  while  still  maintaining  feature  origins. 
The  SubrScene  visual  software  supports  several  database  pager  formats.  These  include  a  fast-page  featureless 
database  format  unique  to  SubrScene,  the  TerraPage  format,  and  a  generic  Large  Area  Tile  Page  pager.  Output  in 
each  of  these  pager  formats  will  be  generated  for  both  testing  and  platform  optimizations. 

Integration  and  development  of  the  MsDB  for  SubrScene  is  being  done  in  three  distinct  passes.  The  first  pass 
involves  testing  the  visual  data  generated  from  RTTC.  OTW  imagery  and  elevation  data  will  be  converted  to  the 
SubrScenePage  (SSPJ  format.  This  database  will  not  include  any  features  or  culture.  Since  this  process  is  relatively 
simple,  it  will  only  take  a  few  hours  to  produce  a  usable  database.  As  with  all  databases,  your  product  is  only  as 
good  as  the  data  you  begin  with. 

The  second  pass  is  designed  to  take  the  PTN  database  product  and  apply  visual  textures  and  materials  to  rapidly 
produce  a  usable  version  of  the  MsDB.  In  this  pass  the  created  geometry  will  be  modified  using  in-house  tools  to 
search  for  and  replace  select  attributes  of  the  database.  Alterations  in  this  second  pass  will  be  focused  on  Level  of 
Detail  (LQD)  and  materi  al  properties  of  the  data.  In  addition,  textures  will  be  rep  laced  to  give  the  database  the  look 
and  feel  of  the  visual  spectrum.  A  cliptexture  of  real  imagery  will  be  applied  to  cover  the  generic  terrain  skin. 
However,  since  the  complexity  and  resolution  of  data  used  by  PTN  is  extremely  high,  performance  for  this  second 
cut  will  likely  be  an  issue.  Essentially,  the  product  will  be  a  geometric  clone  of  the  PTN  database  with  visual 
attributes  and  LOD  scaling  added  to  better  approximate  the  visual  spectrum.  Cutting  areal  features  into  the  terrain 
skin  will  complete  the  second  pass  of  the  database  generation  process. 

The  first  two  passes  of  this  process  serve  primarily  to  quickly  produce  usable  databases  for  the  visual  system.  The 
third  and  final  pass  will  generate  a  product  directly  from  the  source  data.  COTS  products  such  as  TerraVista  and 
MultiGen  will  be  used  to  combine  visual  imagery,  elevation  data,  and  cultural  features  registered  by  RTC  into  a 
platform-targeted  database.  The  level  of  fidelity  for  the  COTS  generated  visual  database  will  be  selected  based  on 
the  target  platform  and  the  simulation  requirements.  Outputs  from  both  products  arc  supported  in  SubrScene  and  the 
process  of  taking  that  generated  data  to  run-time  is  trivial. 


8.  Current  Status  and  Road  Ahead 


8 A  PTN 


At  the  time  of  this  writing,  the  modules  for  reading  the  feature  and  elevation  data  are  complete.  The  PFB  writer  is 
completed .  And  a  limited  function  terrainGen  core  exists  supporting  roads,  rivers,  and  trees.  The  CTDB  writer  is 
still  in  progress.  The  graphical  user  interface  is  only  in  the  design  stage*  but  an  XML  based  project  file  and 
appropriate  readers  have  been  developed.  The  early  versions  of  the  software  will  run  on  the  command  line  using  a 
hand  generated  XML  project  file  for  user  input.  Additional  integration  is  still  necessary  before  we  have  a 
functioning  version.  In  addition  modules  for  building  and  lake  must  still  be  integrated  into  ihe  core.  We  have 
completed  the  test  build  using  our  existing  toolset. 

8.3  MMRS 

The  MMRS  is  gradually  being  integrated  into  the  AFRL  system.  Refinement  of  SPB  issues  will  continue  to  emerge 
as  new  features  are  added,  unrealistic  radar  effects  identified,  and  as  requirements  evolve.  r!fhe  team  will  address 
these  issues  as  they  are  identified  and  corrections  implemented  wrhere  appropriate.  Current  issues  still  open  are 
shoreline  elevation  anomalies  and  the  further  identification  of  entities  that  are  candidate  for  replacement  with  higher 
fidelity  models.  Transmission  towrers  is  one  candidate  example. 


9*  Summary 

AFRL/VAC  has  developed  a  process  for  constructing  large-area  3-D  muUispcctral  terrain  databases  to  support 
Simulation  Based  R&D  (SBR&D)  concept  development  simulation,  research  and  T&E  activities.  The  Multispectral 
database  (MsDB)  development  effort  was  intended  to  investigate  a  database  development  approach  built  from 
muhispectral/hyperspectral  imagery  and  elevation  data  that  also  incorporates  material  classification  data  capable  of 
supporting  realistic  out-the-window  visual  and  multispectra]  terrain  displays  and  weapons1' sensor  models. 
Integrating  the  MsDB  into  these  legacy  models  enabled  us  to  develop  cockpit  representations  that  immerse  the 
warfighter,  scientist,  analyst,  and  testers  into  a  dynamic  environment  suitable  for  engineering  level  test  and 
evaluation  of  a  multitude  of  systems. 
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